Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Rapid prototypingembeddedsystemsbypython

390 views

Published on

Bit-encoded parser in communication system is not easy to program, and usually written in C. Sometimes for lack of toolchain or reduction of memory footprint, programmers might still need to write it in assembly, and it's definitely not a happy task. The author tries to share his happy experience on rapid prototyping this kind of parser in Python, and later coded them in assembly running in an embedded processor.

This was presented in PyCon APAC 2015.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Rapid prototypingembeddedsystemsbypython

  1. 1. Happy Rapid Prototyping Embedded Assembly by Python: My Experience Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time Systems June 5, 2015 Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 1 / 30
  2. 2. How to Contact Me Contact me at Twitter: @AlbertHuang314 LinkedIn: AlbertHuang314 E-mail address: alberthuang314@gmail.com Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 2 / 30
  3. 3. Outline 1 Introduction 2 Rapid Prototype in Python 3 Converting Python to Assembly, Happily 4 Concluding Remarks Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 3 / 30
  4. 4. Introduction Introduction In a communication system, in order to meet backward compatibility, capabilites of both modems are often needed to exchange before real data transmission. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 4 / 30
  5. 5. Introduction Introduction In a communication system, in order to meet backward compatibility, capabilites of both modems are often needed to exchange before real data transmission. A parser is required to parse those data online. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 4 / 30
  6. 6. Introduction Introduction In a communication system, in order to meet backward compatibility, capabilites of both modems are often needed to exchange before real data transmission. A parser is required to parse those data online. Programmers on a communication system are often required to write code in assembly, in order to reduce memory footprint. Because SRAM occupies a lot of area on a chip! Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 4 / 30
  7. 7. Introduction Exchanging Capabilities: Modem Transmits Root Node1 Node2 Node3 Node21 Node22 N31 N32 Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 5 / 30
  8. 8. Introduction Exchanging Capabilities: Modem Receives Root Node1 Node2 Node3 Node21 Node22 Node23 N31 N32 N33 N34 Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 6 / 30
  9. 9. Introduction Exchanging Capabilities: Modem Send the Final Decision Root Node1 Node21 N31 N32 Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 7 / 30
  10. 10. Introduction Tree Encoding These trees are encoded into an octet (byte) stream. Some bits are used to mark the end of different levels. Every leaf is required to parse even you don’t support it. The parser need to parse it online so that it knows where the tree is ended. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 8 / 30
  11. 11. Introduction So Let’s Code in Assembly Directly! No kidding! Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 9 / 30
  12. 12. Introduction So Let’s Code in Assembly Directly! No kidding! Normally we will code in MATLAB first for signal processing functions. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 9 / 30
  13. 13. Introduction So Let’s Code in Assembly Directly! No kidding! Normally we will code in MATLAB first for signal processing functions. But MATLAB is totally useless in building this parser. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 9 / 30
  14. 14. Introduction So Let’s Code in Assembly Directly! No kidding! Normally we will code in MATLAB first for signal processing functions. But MATLAB is totally useless in building this parser. Should I code in C first as other engineers do? Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 9 / 30
  15. 15. Introduction So Let’s Code in Assembly Directly! No kidding! Normally we will code in MATLAB first for signal processing functions. But MATLAB is totally useless in building this parser. Should I code in C first as other engineers do? I decided to give Python a try in order to find the best architecture. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 9 / 30
  16. 16. Introduction Let’s Make a Rapid Prototype First At the time of writing this parser, I had been a full-time assembly programmer for at least four years. that was five or six years ago! Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 10 / 30
  17. 17. Introduction Let’s Make a Rapid Prototype First At the time of writing this parser, I had been a full-time assembly programmer for at least four years. that was five or six years ago! I wrote assembly every day, for four years! Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 10 / 30
  18. 18. Introduction Let’s Make a Rapid Prototype First At the time of writing this parser, I had been a full-time assembly programmer for at least four years. that was five or six years ago! I wrote assembly every day, for four years! This code will be eventually maintained by other programmers at least I wish when I started to write it maintainability is important Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 10 / 30
  19. 19. Introduction Let’s Make a Rapid Prototype First At the time of writing this parser, I had been a full-time assembly programmer for at least four years. that was five or six years ago! I wrote assembly every day, for four years! This code will be eventually maintained by other programmers at least I wish when I started to write it maintainability is important So I need to experiment different architecture for simplicity, clarity, and maintainability. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 10 / 30
  20. 20. Rapid Prototype in Python Rapid Prototype Rapid prototyping is quickly building a scale model and (sometimes) will be dumped later. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 11 / 30
  21. 21. Rapid Prototype in Python How to Rapid Prototype for Assembly? I had done rapid prototyping for C many times, and I enjoy doing that in Python. Because it actually saves a lot of time for debugging Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 12 / 30
  22. 22. Rapid Prototype in Python How to Rapid Prototype for Assembly? I had done rapid prototyping for C many times, and I enjoy doing that in Python. Because it actually saves a lot of time for debugging But how to apply this to assembly? Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 12 / 30
  23. 23. Rapid Prototype in Python How to Rapid Prototype for Assembly? I had done rapid prototyping for C many times, and I enjoy doing that in Python. Because it actually saves a lot of time for debugging But how to apply this to assembly? After so many years of writing assembly, I realized that C is actually a thin wrapper of assembly. what can be applied to C can also be easily applied to assembly Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 12 / 30
  24. 24. Rapid Prototype in Python Characteristics of Assembly Programming on DSP Bit-wise arithmetic or branching is common Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 13 / 30
  25. 25. Rapid Prototype in Python Characteristics of Assembly Programming on DSP Bit-wise arithmetic or branching is common Function pointers are used a lot Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 13 / 30
  26. 26. Rapid Prototype in Python Characteristics of Assembly Programming on DSP Bit-wise arithmetic or branching is common Function pointers are used a lot Word-addressing Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 13 / 30
  27. 27. Rapid Prototype in Python Characteristics of Assembly Programming on DSP Bit-wise arithmetic or branching is common Function pointers are used a lot Word-addressing All variables are global Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 13 / 30
  28. 28. Rapid Prototype in Python Characteristics of Assembly Programming on DSP Bit-wise arithmetic or branching is common Function pointers are used a lot Word-addressing All variables are global And I prefer table-driven programming Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 13 / 30
  29. 29. Rapid Prototype in Python Design Considerations Code size must be as small as possible Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 14 / 30
  30. 30. Rapid Prototype in Python Design Considerations Code size must be as small as possible Easy to maintain, and that means: easy to add handlers of leaves in assembly by other engineers you don’t need to trace parsing engine very often Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 14 / 30
  31. 31. Rapid Prototype in Python Design Considerations Code size must be as small as possible Easy to maintain, and that means: easy to add handlers of leaves in assembly by other engineers you don’t need to trace parsing engine very often If you need to trace parser engine, it should let you grab the idea very fast. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 14 / 30
  32. 32. Rapid Prototype in Python Python Code Bit-wise arithmetic It’s not so easy to perform bit-wise directly in Python. So I use True/False list instead Very easy to follow and convert later Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 15 / 30
  33. 33. Rapid Prototype in Python Python Code Function Pointers In Python, just store the name of functions in a list def a ( t e s t ) : print ( t e s t ) L i s t = [ a ] L i s t [ 0 ] ( ’ argument ’ ) Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 16 / 30
  34. 34. Rapid Prototype in Python Python Code Global Variables That’s not a problem in Python Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 17 / 30
  35. 35. Rapid Prototype in Python Tree Parser Architecture This is the most important part I’ve built several prototypes for this tree parsing/leaf handling It’s very easy to experiment in Python And I need to know which part will be changed very often those parts will be modified by other DSP programmers a lot and need to be as simple as possible Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 18 / 30
  36. 36. Rapid Prototype in Python Tree Parser Architecture Parser Engine Stack or not stack, that is the question! Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 19 / 30
  37. 37. Rapid Prototype in Python Tree Parser Architecture Parser Engine Stack or not stack, that is the question! Number of the first level nodes are fixed. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 19 / 30
  38. 38. Rapid Prototype in Python Tree Parser Architecture Parser Engine Stack or not stack, that is the question! Number of the first level nodes are fixed. Finally I decided not to use stack for simplicity. It’s easier to maintain for other programmers or me! Everytime when I have to debug this part, Python code plays an important role for me to pick up again. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 19 / 30
  39. 39. Rapid Prototype in Python Tree Parser Architecture Leaf Handlers This part will be added/modified by other engineers very often. So I tried to make it as simple as possible. If transmit mode, generate octets for this leaf. If receive mode, compare and save these octets with our capabilities. Then insert this handler subroutine to the table, according to its tree position. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 20 / 30
  40. 40. Converting Python to Assembly, Happily Every Program Is Created Twice Every Program is created twice, first in Python, then in assembly Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 21 / 30
  41. 41. Converting Python to Assembly, Happily Converting Python to Assembly, Happily Every experienced assembly programmer has his/her own design pattern. When we see the program control loop, we know how to code in assembly. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 22 / 30
  42. 42. Converting Python to Assembly, Happily Assembly Design Pattern If-then-else Bit test a flag, and branch to else part if the bit is clear Do the “then” part Branch to the end Else: Do the “else” part The end: Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 23 / 30
  43. 43. Converting Python to Assembly, Happily Assembly Design Pattern Switch case Table: Table-Driven Switch Case Value Subroutine Address 0 Subroutine0 1 Subroutine1 2 Subroutine2 3 Subroutine3 4 Subroutine4 0xFFFF End marker Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 24 / 30
  44. 44. Converting Python to Assembly, Happily Assembly Design Pattern Leaf Handlers This structure is consistent across all leaf handlers: register r0 as stored octet stream buffer register r1 as input/output octet stream If transmit mode, generate octets from r0 and write to the buffer pointed by r1 If receive mode, receive octets from r1 and compare with r0, write to r0 Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 25 / 30
  45. 45. Converting Python to Assembly, Happily Assembly Design Pattern And so on. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 26 / 30
  46. 46. Converting Python to Assembly, Happily Assembly Design Pattern And so on. I can’t disclose too much detail. Because IC design house is conservative. And the memory is quite vague to me right now because I don’t have access those code for two years. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 26 / 30
  47. 47. Converting Python to Assembly, Happily Assembly Design Pattern And so on. I can’t disclose too much detail. Because IC design house is conservative. And the memory is quite vague to me right now because I don’t have access those code for two years. But the basic idea is illustrated. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 26 / 30
  48. 48. Converting Python to Assembly, Happily The End of Story I was like a typist during that period of converting python to assembly, expensive one. More than 4k word of assembly were finished in a week or two. That is fast for writing assembly code. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 27 / 30
  49. 49. Converting Python to Assembly, Happily So What Part Is Happy Part? Coding in Python Happy Converting from Python to assembly like a typist Happy Python code was used as a offline parser later Happy Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 28 / 30
  50. 50. Concluding Remarks Concluding Remarks Coding in assembly is challenging. Coding in assembly without a plan is disaster. When you design something new and you want to experiment different software architecutre in assembly, Python is extremely useful. Python is a very good rapid prototyping building tool for assembly After so many years, I can still feel how happy I was at building those rapid prototype and assembly. Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 29 / 30
  51. 51. Concluding Remarks Q & A Albert Chun-Chieh Huang Principal Firmware Engineer on Real-time SystemsHappy Rapid Prototyping Embedded Assembly by Python: My ExperienceJune 5, 2015 30 / 30

×