• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Issta11
 

Issta11

on

  • 4,520 views

slides from John Regehr's keynote at ISSTA 2011

slides from John Regehr's keynote at ISSTA 2011

Statistics

Views

Total Views
4,520
Views on SlideShare
816
Embed Views
3,704

Actions

Likes
3
Downloads
16
Comments
0

14 Embeds 3,704

http://blog.regehr.org 3648
http://news.int80.biz 25
http://dashboard.bloglines.com 7
http://feeds.feedburner.com 6
http://www.hanrss.com 3
http://newsblur.com 3
http://translate.googleusercontent.com 3
http://webcache.googleusercontent.com 2
http://feeds.churchturing.org 2
http://www.ljr-embed.ru 1
http://www.netvibes.com 1
http://snap.diclonius.org 1
https://twitter.com 1
http://xianguo.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Issta11 Issta11 Presentation Transcript

    • Testing EmbeddedSoftware
      John Regehr
      University of Utah
    • “Over 15 billion ARM based chips shipped to date”
      [ARM web site, 2011]
      “The microcontroller market is forecast to reach over $16 billion worldwide in 2011”
      [Microcontroller Market Tracker, 2011]
      2
    • 3
    • Diverse!
      4
    • Diverse!
      I have 6 pins and 32 bytes of RAM
      5
    • Diverse!
      6
    • Diverse!
      7
    • Diverse!
      I am quad core @ 1.5 GHz and have a GPU
      8
    • Usually there are multiple processors
      On-chip networks
      In-device networks
      Distributed systems
      Resource constraints are…
      Severe – to minimize unit cost
      Hard – failure if system runs out of…
      Time
      RAM – stack or heap
      Energy
      9
    • Continuously interact with the world through I/O devices
      May be little abstraction of HW
      Probably using both interrupt handlers and threads
      Often there are fault tolerance and security requirements
      10
    • Sensor network -> 103–105 LOC
      Modern airplane -> 106–107 LOC
      Hybrid vehicle -> 107–108 LOC
      How do we get these right?
      Mostly testing
      11
    • Software on many individual processors is small
      Permits aggressive analysis and testing
      Constrained domain simplifies testing
      Embedded systems are (by definition) special-purpose devices
      12
    • The “Real System Problem”
      Many interesting embedded codes are proprietary
      Necessary tools may be expensive or nonexistent
      Compilers, debuggers, simulators
      May not be able to run it in the lab
      Often lacks specifications and oracles
      13
    • Consequently, academic embedded work may be…
      Forced to use small, contrived examples
      Out of tune with industry
      14
    • Consequently, academic embedded work may be…
      Forced to use small, contrived examples
      Out of tune with industry
      15
      Solution:
      Ubiquitous open embedded platforms
    • Arduino
      Arduino Uno:
      8-bit AVR processor @ 16 MHz
      2 KB RAM
      ~$30
      Emphasis is on interfacing
      16
    • Arduino
      Nice IDE + libraries + C/C++
      Minimal abstraction of the embedded processor
      18 new books in 2011
      17
      • Simulators and model checkers for AVR code exist
      • Very few Arduino tool papers exist
      • This is a big opportunity
      Arduino
      Nice IDE + libraries + C/C++
      Minimal abstraction of the embedded processor
      18 new books in 2011
      18
    • TinyOS
      OS and middleware support for sensor networks
      Sensing
      Collection and dissemination
      Localization
      Applications are in nesC, a C dialect
      19
    • TinyOS
      “Motes” based on a variety of MCUs
      Cost $50 – $200
      Good simulators exist
      There are a few books
      20
    • TinyOS
      • ~100 tool papers
      • Many open problems
      “Motes” based on a variety of MCUs
      Cost $50 – $200
      Good simulators exist
      There are a few books
      21
    • Android
      OS + middleware for smart phones / tablets
      ARM based hardware running Linux
      Much less constrained than motes and Arduino
      22
    • Android
      Application code in Java
      Great tools
      Tons of books
      23
    • Android
      • < 100 tool papers
      • Most are very recent
      • This is not a scary platform
      Application code in Java
      Great tools
      Tons of books
      24
    • ROS – Robot Operating System
      Linux-based infrastructure for programming robots
      Primary abstraction is graph of communicating processes
      Local and distributed
      25
    • ROS – Robot Operating System
      • Very few ROS tool papers exist
      Linux-based infrastructure for programming robots
      Primary abstraction is graph of communicating processes
      Local and distributed
      26
    • Plenty of other open embedded platforms exist
      FreeRTOS
      Contiki
      Pacemaker Challenge
      Etc.
      Embarrassment of riches
      Still, huge room for improvement
      Where’s the open automobile?
      27
    • So, let’s test some embedded software
      But what are we testing for?
      28
    • Properties / Oracles
      Temporal safety
      Deadlines
      Or just responsiveness
      Memory safety
      Contracts / assertions
      Reference implementation
      29
    • Worst-Case Execution Time
      What is the upper bound on execution time for a piece of code?
      We care because the world has deadlines
      Static analysis of WCET is extremely difficult if there is…
      A cache
      Preemption
      An aggressive processor
      30
    • True WCET
      Number of executions
      Execution time
      Conservative WCET
      Longest observed ET #2
      Longest observed ET #1
      31
    • 32
      +
      =
      printf()
      pthread_attr_setstacksize (&attr, &mystacksize);
      HANDLE WINAPI CreateThread(
      LPSECURITY_ATTRIBUTES lpThreadAttributes,
      SIZE_T dwStackSize,
      LPTHREAD_START_ROUTINE lpStartAddress,
      LPVOID lpParameter,
      DWORD dwCreationFlags,
      LPDWORD lpThreadId );
    • Stack Overflow in TinyOS
      33
      4 KB
    • Stack Overflow in TinyOS
      34
      main()
      4 KB
    • Stack Overflow in TinyOS
      35
      main()
      irq 0
      4 KB
    • Stack Overflow in TinyOS
      36
      main()
      irq 0
      4 KB
      irq 1
    • Stack Overflow in TinyOS
      37
      main()
      irq 0
      4 KB
      irq 1
    • Stack Overflow in TinyOS
      38
      Not the same thing as buffer overflow!
      Type safe language doesn’t solve this problem
      main()
      irq 0
      4 KB
      irq 1
    • Eliminating Stack Overflow
      Testing is hard
      Need to drive code to its WC stack depth
      Interrupt coincidences are rare
      Approach: Static analysis of compiled code
      Can’t estimate stack depth of source
      39
    • Estimate WC stack depth of each sequential flow, handling
      Indirect branches
      Recursion
      Loads into the stack pointer
      Compute “interrupt preemption graph”
      Find longest cycle in this graph
      40
    • 41
      in r24, 0x3f ; r24 <- CPU status
      register
      cli ; disable interrupts
      adc r24, r24 ; carry bit <- prev
      interrupt status
      eor r24, r24 ; r24 <- 0
      adc r24, r24 ; r24 <- carry bit
      mov r18, r24 ; r18 <- r24
      ... critical section ...
      and r18, r18 ; test r18 for zero
      breq .+2 ; if zero, skip next
      instruction
      sei ; enable interrupts
      ret ; return from function
    • Stack analysis tool deployed in the TinyOS distribution
      Results are typically much larger than worst observed stack depths
      But, we validated its results by randomly firing interrupts
      42
    • Need… more… oracles…
      43
    • TinyOS applications are built using components
      Interface requirements documented but not checked
      Interface misuse often silent
      44
    • We augmented nesC with contracts
      Dynamic checking reasonable efficient
      Found some long-standing bugs
      45
    • nesC is not type safe
      Memory safety bugs in TinyOS are difficult
      We ported an existing safe C dialect
      Found some otherwise-impossible bugs
      Main problem was getting overhead under control
      Whole-program optimization
      46
    • 47
      Code size
      35%
      13%
      -11%
    • 48
      Increasing Availability
      Normal TinyOS:
      0% average
      availability
      Array
      Out-of-bounds
      Normal
      TinyOS
      Safe TinyOS:
      95% average
      availability
      Array
      Out-of-bounds
      Rebuild
      Soft state
      Safe
      TinyOS
      Reboot
    • What about application-level sensornet properties?
      All the interesting ones are distributed
      We adapted TOSSIM, a non-cycle-accurate simulator, to be…
      A random tester
      A depth-bounded model checker
      Oracles:
      Type safety checks
      Application-level properties
      49
    • Application-Level Properties
      Eventually…
      Each send buffer is unlocked
      No cycles in the routing tree
      All nodes become part of the collection tree
      All nodes have consistent values
      6 out of 8 of these properties require global knowledge
      50
    • Found 12 previously unknown bugs in TinyOS 2.0
      10 safety, 2 liveness
      Random testing outperformed depth-bounded model checking
      Even after a lot of work on POR
      But required work to shorten long error traces
      51
    • Conclusions
      Open embedded platforms exist
      Some have steep learning curves
      Finding oracles is hard
      Generating valid input is hard
      Embedded systems are fun and important and rewarding
      52