Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

[PyCon Korea 2018] Python in Chains: Running Python Code for IoT Projects on a custom-built Blockchain Network

645 views

Published on

This talk covers the journey of running Python code as a blockchain (Ethereum) Smart Contract by implementing a modified Python runtime environment / Ethereum Virtual Machine software, and building a blockchain based home device network with the modified runtime and Smart Contract code written in Python.

본 발표는 파이썬 코드를 Ethereum 블록체인에서 구동될 수 있는 Smart Contract로 구동할 수 있도록 하는 삽질의 여정을 다룹니다. 보다 상세하게는, 상기의 목적에 맞도록 수정된 파이썬 런타임과 Ethereum Virtual Machine (EVM) 소프트웨어를 구현하고, 이를 사용하여 블록체인 기반의 가정 IoT 네트워크 위에서 파이썬 언어로 작성된 Smart Contract 코드를 작성하는 것에 관하여 다룹니다.

Published in: Technology
  • Be the first to comment

[PyCon Korea 2018] Python in Chains: Running Python Code for IoT Projects on a custom-built Blockchain Network

  1. 1. PYTHON IN CHAINS RUNNING PYTHON CODE FOR IOT PROJECTS ON A CUSTOM-BUILT BLOCKCHAIN NETWORK Daniel Hong (홍석현) | @unifiedh
  2. 2. “I KNOW THERE HAS BEEN A LOT OF HYPE SURROUNDING BLOCKCHAIN TECH, BUT… PYTHON ON THE BLOCKCHAIN? ISN’T THAT LIKE…”
  3. 3. Photo courtesy of: Hyunse Lee “지역주민과 함께 읽는 한국학: 파이썬으로 읽는 디지털 인문학” “Korean Studies with local residents: Learning Digital Humanities using Python”
  4. 4. …NOPE.
  5. 5. 1. Python is a universal language that can run on almost every computer. SOME BASIC FACTS: 2. Blockchain platforms can also act as a computer, and be programmed like any other computer. 3. Python and some blockchain platforms share a similarly structured runtime architecture.
  6. 6. BLOCKCHAIN PLATFORMS AS A COMPUTER Ethereum Virtual Machine (EVM)
  7. 7. BLOCKCHAIN PLATFORMS AS A COMPUTER Ethereum Virtual Machine (EVM) Provides a Turing-complete execution environment Provides a high level programming language & easier code deployment support Adjustable code execution fees (Gas) Better blockchain extensibility for application usage
  8. 8. CODE EXECUTION ON THE EVM pragma solidity ^0.4.11; contract hello { string world; constructor() public { world = "Hello, World!"; } function sayHello() public view returns (string) { return world; } } Solidity Code (Smart Contract)
  9. 9. CODE EXECUTION ON THE EVM pragma solidity ^0.4.11; contract hello { string world; constructor() public { world = "Hello, World!"; } function sayHello() public view returns (string) { return world; } } Solidity Code (Smart Contract) Ethereum Blockchain Compile & Deploy
  10. 10. CODE EXECUTION ON THE EVM pragma solidity ^0 .4 .1 1 ; contract hello { string world; constructor() public { world = "Hello, World!"; } function sayHello() public view returns (string) { return world; } } Solidity Code (Smart Contract) Ethereum Blockchain Compile & Deploy Application Code Smart Contract Interaction Library Access Smart Contract via Contract ABI
  11. 11. BLOCKCHAIN PLATFORMS AS A COMPUTER Ethereum Virtual Machine (EVM) Poor development environment support Confusing language syntax Inefficient Gas system (The Extensibility Issue) Language compatibility Resource-intensive; not suitable for embedded systems
  12. 12. BLOCKCHAIN PLATFORMS AS A COMPUTER Ethereum Virtual Machine (EVM) Poor development environment support Confusing language syntax Inefficient Gas system (The Extensibility Issue) Language compatibility Resource-intensive; not suitable for embedded systems
  13. 13. PYTHON TO THE RESCUE
  14. 14. PYTHON TO THE RESCUE Integrating Smart Contracts with existing Python code and applications Writing & Compiling Ethereum Smart Contracts with Python Modifying & Running the EVM on embedded devices for IoT system development
  15. 15. PYTHON TO THE RESCUE Integrating Smart Contracts with existing Python code and applications Writing & Compiling Ethereum Smart Contracts with Python Modifying & Running the EVM on embedded devices for IoT system development
  16. 16. SMART CONTRACT INTEGRATION ‣ web3.py: A Python implementation of the web3.js library (originally written in JavaScript); used to call Smart Contract code within local Python code ‣ Populus: A Python-based Smart Contract development environment for Ethereum blockchain developers using Python ‣ Py-EVM: A Python implementation of the Ethereum Virtual Machine (EVM) itself; will modify this to build a custom implementation of the EVM optimized for embedded systems SOME PREREQUISITES: ‣ Trinity: An Ethereum blockchain client based on the Py-EVM project & completely written in Python
  17. 17. SMART CONTRACT INTEGRATION INSTALLING REQUIRED SOFTWARE & LIBRARIES # Install solc & py-solc $ brew update; brew upgrade; brew tap ethereum/ethereum; brew install solidity $ python3 -m solc.install v0.4.11 # Install Populus $ brew install pkg-config libffi autoconf automake libtool openssl $ pip3 install populus # Install Trinity $ pip3 install -U trinity
  18. 18. SMART CONTRACT INTEGRATION WRITING A “HELLO WORLD” SMART CONTRACT & INTEGRATING SOLIDITY CODE WITH PYTHON pragma solidity ^0.4.11; contract hello { string world; constructor() public { world = "Hello, World!"; } function sayHello() public view returns (string) { return world; } } Solidity Code (Smart Contract) ./contracts/hello.sol
  19. 19. SMART CONTRACT INTEGRATION WRITING A “HELLO WORLD” SMART CONTRACT & INTEGRATING SOLIDITY CODE WITH PYTHON pragma solidity ^0.4.11; contract hello { string world; constructor() public { world = "Hello, World!"; } function sayHello() public view returns (string) { return world; } } Solidity Code (Smart Contract) ./contracts/hello.sol Python Code ./tests/test_HelloWorld.py def test_HelloWorld(chain): hi, _ = chain.provider.get_or_deploy_contract(hello) helloString = hi.call().sayHello() assert helloString == “Hello, World!” #Test Only
  20. 20. def test_HelloWorld(chain): hi, _ = chain.provider.get_or_deploy_contract(hello) helloString = hi.call().sayHello() assert helloString == “Hello, World!” #Test Only SMART CONTRACT INTEGRATION WRITING A “HELLO WORLD” SMART CONTRACT & INTEGRATING SOLIDITY CODE WITH PYTHON pragma solidity ^0.4.11; contract hello { string world; constructor() public { world = "Hello, World!"; } function sayHello() public view returns (string) { return world; } } Solidity Code (Smart Contract) ./contracts/hello.sol Python Code ./tests/test_HelloWorld.py 1. chain argument: py.test fixture, provided by populus’ pytest plugin - chain object provided by populus, creates & calls a temp blockchain (“tester”) on memory 2. Creates a web3 contract object instance that corresponds to the hello contract written in Solidity - The hi object created here acts as a wrapper for Solidity code for use within Python 3. Call the sayHello() function defined within the hello contract 4. (Used with pytest) Check whether the hello contract was executed successfully
  21. 21. SMART CONTRACT INTEGRATION TESTING SMART CONTRACT INTEGRATION WITH PYTEST # Create a new Populus Project $ mkdir helloTest $ cd helloTest $ populus init $ rm contracts/Greeter $ rm tests/test_greeter.py # Copy Solidity & Python source code $ cp ~/sources/hello.sol contracts/hello.sol $ cp ~/sources/test_HelloWorld.py tests/test_HelloWorld.py # Compile & Deploy on a test blockchain $ populus compile $ populus deploy —chain tester hello # Run pytest $ py.test
  22. 22. SMART CONTRACT INTEGRATION TESTING SMART CONTRACT INTEGRATION WITH PYTEST $ populus compile
  23. 23. SMART CONTRACT INTEGRATION TESTING SMART CONTRACT INTEGRATION WITH PYTEST $ populus deploy —chain tester hello
  24. 24. SMART CONTRACT INTEGRATION TESTING SMART CONTRACT INTEGRATION WITH PYTEST $ py.test
  25. 25. PYTHON TO THE RESCUE Integrating Smart Contracts with existing Python code and applications Writing & Compiling Ethereum Smart Contracts with Python Modifying & Running the EVM on embedded devices for IoT system development
  26. 26. PRIVATE BLOCKCHAIN SETUP FOR IOT Private Ethereum Blockchain Controller Smart Contract Django Controller Server Web Browser Mobile App Raspberry Pi Raspberry Pi Raspberry Pi Raspberry Pi Coffee Machine Curtains Light Controllers A/C
  27. 27. PRIVATE BLOCKCHAIN SETUP FOR IOT Private Ethereum Blockchain Controller Smart Contract Django Controller Server Web Browser Mobile App Raspberry Pi Raspberry Pi Raspberry Pi Raspberry Pi Coffee Machine Curtains Light Controllers A/C ?
  28. 28. EVM CODE EXECUTION OVERVIEW THE ETHEREUM BLOCKCHAIN Block N Header: Previous Block Hash Nonce Timestamp Uncle Hash Beneficiary Logs Bloom PoW Difficulty Extra Data Block Number Gas Limit Used Gas Amount Mix Hash State Root Transaction Root Receipt Root Block N+1 Header: Previous Block Hash Nonce Timestamp Uncle Hash Beneficiary Logs Bloom PoW Difficulty Extra Data Block Number Gas Limit Used Gas Amount Mix Hash State Root Transaction Root Receipt Root Receipt Merkle Tree Transaction Merkle Tree Receipt Merkle Tree Transaction Merkle Tree Nonce Code Hash Balance Storage Root Code Nonce Code Hash Balance Storage Root Merkle-Patricia State Trie for block N Merkle-Patricia State Trie for block N+1 73 95 Uncle Block Headers List Uncle Block Headers List
  29. 29. THE ETHEREUM BLOCKCHAIN Block N Header: Previous Block Hash Nonce Timestamp Uncle Hash Beneficiary Logs Bloom PoW Difficulty Extra Data Block Number Gas Limit Used Gas Amount Mix Hash State Root Transaction Root Receipt Root Block N+1 Header: Previous Block Hash Nonce Timestamp Uncle Hash Beneficiary Logs Bloom PoW Difficulty Extra Data Block Number Gas Limit Used Gas Amount Mix Hash State Root Transaction Root Receipt Root Receipt Merkle Tree Transaction Merkle Tree Receipt Merkle Tree Transaction Merkle Tree Nonce Code Hash Balance Storage Root Code Nonce Code Hash Balance Storage Root Merkle-Patricia State Trie for block N Merkle-Patricia State Trie for block N+1 73 95 Uncle Block Headers List Uncle Block Headers List EVM CODE EXECUTION OVERVIEW
  30. 30. THE ETHEREUM VIRTUAL MACHINE: AN OVERVIEW (MEMORY MANAGEMENT) ‣ Memory: A temporary storage area used when calling functions, or when during regular memory operations ‣ Stack: No concepts of “registers” are provided; a virtual stack is used instead for operations such as opcode parameters ‣ Storage: A permanent key-value storage mapping 256-bit words to 256-bit words; memory space declared outside user-defined functions & within the contract context are stored here Reference: https://github.com/comaeio/porosity/wiki/Ethereum-Internals EVM CODE EXECUTION OVERVIEW
  31. 31. THE ETHEREUM VIRTUAL MACHINE: OPCODES & ASSEMBLY ‣ The EVM is (theoretically) language independent; however, due to its architectural limitations the EVM cannot take full advantage of all features present in modern programming languages & requires some unique language syntax (to access unique EVM features) ‣ The EVM implements a simple instruction set to make up EVM bytecode (to ensure platform / ISA independence) ‣ The EVM can also be directly programmed at a low-level using an Assembly Language provided by Solidity; Assembly can be used along with Solidity code with inline Assembly EVM CODE EXECUTION OVERVIEW
  32. 32. THE ETHEREUM VIRTUAL MACHINE: OPCODES & ASSEMBLY ‣ The EVM is (theoretically) language independent; however, due to its architectural limitations the EVM cannot take full advantage of all features present in modern programming languages & requires some unique language syntax (to access unique EVM features) ‣ The EVM can also be directly programmed at a low-level using an Assembly Language provided by Solidity; Assembly can be used along with Solidity code with inline Assembly EVM CODE EXECUTION OVERVIEW ‣ The EVM implements a simple instruction set to make up EVM bytecode (to ensure platform / ISA independence)
  33. 33. Reference: https://ethereum.github.io/yellowpaper/paper.pdf
  34. 34. Reference: http://unpyc.sourceforge.net/Opcodes.html
  35. 35. WILL IT BE POSSIBLE TO DEVELOP A SMART CONTRACT USING PYTHON?
  36. 36. SOME METHODS TO PORTING PYTHON TO THE EVM 2. ADDING EVM-SPECIFIC SYNTAX TO PYTHON AS A SUPERSET OF THE PYTHON LANGUAGE 3. (OUR APPROACH) ALLOWING PYTHON CODE TO COMPILE DIRECTLY TO EVM BYTECODE WITHOUT LARGELY MODIFYING OR ADDING PYTHON LANGUAGE SYNTAX 1. IMPLEMENTING THE PYTHON INTERPRETER ITSELF AS A SMART CONTRACT IN SOLIDITY
  37. 37. SOME METHODS TO PORTING PYTHON TO THE EVM 1. IMPLEMENTING THE PYTHON INTERPRETER ITSELF AS A SMART CONTRACT IN SOLIDITY ‣ However, as the EVM is a distributed execution environment, this method may consume excessive compute power & Gas (which is very inefficient in terms of extensibility) ‣ Jython (the JVM implementation of Python) uses this method to run Python code on the JVM (Reimplementing the Python language with Java, rather than C as used by standard CPython) ‣ Because this method abstracts the language compilation process through a modular implementation, existing pure Python code is almost completely reusable
  38. 38. SOME METHODS TO PORTING PYTHON TO THE EVM ‣ The problem: adding new syntax to Python means backwards compatibility cannot be ensured with existing Python code (Can Python with added custom syntax be still considered as Python?) ‣ Similar to how Objective-C added object-oriented programming capabilities with added syntax while also maintaining full backwards compatibility with C ‣ The Ethereum Foundation chose a similar approach with a new language called Vyper, which also targets the EVM (like Solidity) but with Pythonic syntax 2. ADDING EVM-SPECIFIC SYNTAX TO PYTHON AS A SUPERSET OF THE PYTHON LANGUAGE
  39. 39. OUR SOLUTION: A FULL PYTHON PORT BUILDING A PYTHON COMPILER THAT TARGETS THE EVM pragma solidity ^0.4.11; contract Wallet { mapping (address => uint) public balance; function deposit() { balance[msg.sender] += 1; } function withdraw(uint value) { if(balance[msg.sender] < value) throw; balance[msg.sender] -= 1; if(!msg.sender.call.value(value)()) throw; } } Solidity Code (Smart Contract)
  40. 40. OUR SOLUTION: A FULL PYTHON PORT BUILDING A PYTHON COMPILER THAT TARGETS THE EVM pragma solidity ^0.4.11; contract Wallet { mapping (address => uint) public balance; function deposit() { balance[msg.sender] += 1; } function withdraw(uint value) { if(balance[msg.sender] < value) throw; balance[msg.sender] -= 1; if(!msg.sender.call.value(value)()) throw; } } Solidity Code (Smart Contract) Python Code (Smart Contract) import ethereum as eth class Wallet(eth.contract): balance = eth.mapping(eth.address, int) def deposit(): balance[eth.msg.sender] += 1 def withdraw(value: int): if balance[eth.msg.sender] < value: raise ValueError balance[eth.msg.sender] -= 1 if not eth.msg.sender.call.value(value)(): raise ValueError
  41. 41. OUR SOLUTION: A FULL PYTHON PORT BUILDING A PYTHON COMPILER THAT TARGETS THE EVM Python Source Code Bytecode Translation Python Bytecode (.pyc) Python Virtual Machine Library Modules Code Execution A standard CPython implementation of Python
  42. 42. OUR SOLUTION: A FULL PYTHON PORT BUILDING A PYTHON COMPILER THAT TARGETS THE EVM Python Source Code Grammar Python Bytecode (.pyc) Python Virtual Machine Library Modules Code Execution A standard CPython implementation of Python Lexer Parser Compiler
  43. 43. OUR SOLUTION: A FULL PYTHON PORT BUILDING A PYTHON COMPILER THAT TARGETS THE EVM Python Source Code On-the-fly EVM Bytecode Translation EVM Bytecode Modified EVM Code Execution Precompiled EVM bytecode (Imported as Python Library) Deploy Our Python-EVM Implementation Structure
  44. 44. OUR SOLUTION: A FULL PYTHON PORT BUILDING A PYTHON COMPILER THAT TARGETS THE EVM Python Source Code EVM Bytecode Code Execution Our Python-EVM Implementation Structure Precompiled EVM bytecode (Imported as Python Library) Deploy Grammar (CPython) Modified Lexer (Added EVM-specific tokens) Modified Parser (Generates Solidity-style structure) Compiler with solc backend Modified EVM
  45. 45. OUR SOLUTION: A FULL PYTHON PORT BUILDING A PYTHON COMPILER THAT TARGETS THE EVM pragma solidity ^0.4.11; contract Wallet { mapping (address => uint) public balance; function deposit() { balance[msg.sender] += 1; } function withdraw(uint value) { if(balance[msg.sender] < value) throw; balance[msg.sender] -= 1; if(!msg.sender.call.value(value)()) throw; } } Solidity Code (Smart Contract) Python Code (Smart Contract) import ethereum as eth class Wallet (eth.contract): balance = eth.mapping(eth.address, int) def deposit(): balance[eth.msg.sender] += 1 def withdraw(value: int): if balance[eth.msg.sender] < value: raise ValueError balance[eth.msg.sender] -= 1 if not eth.msg.sender.call.value(value)(): raise ValueError 1. ethereum module: A Python wrapper that allows EVM-specific features to be called within Python code 2. Wallet class: inherits the contract class defined within the ethereum module to define a Contract 3. eth.mapping: returns an access token to the Mapping type value ; eth.address: same as the address type defined with Solidity (20 byte value) 4. Modified syntax to access Mapping type variable
  46. 46. OUR SOLUTION: A FULL PYTHON PORT DYNAMIC PYTHON-STYLE ETHEREUM VIRTUAL MACHINE Python Source Code EVM Bytecode Code Execution Our Python-EVM Implementation Structure Precompiled EVM bytecode (Imported as Python Library) Grammar (CPython) Modified Lexer (Added EVM-specific tokens) Modified Parser (Generates Solidity-style structure) Compiler with solc backend Modified EVM Deploy
  47. 47. ‣ …but I cannot disclose all of them in detail just yet ‣ Execution Logic Improvements: No simultaneous code execution on devices marked as “embedded”; only codehash checks (integrity insurance) are performed on-device ‣ AOT Compiler: Allows ahead-of-time EVM bytecode compilation with embedded devices OUR SOLUTION: A FULL PYTHON PORT DYNAMIC PYTHON-STYLE ETHEREUM VIRTUAL MACHINE
  48. 48. ONE MORE THING…
  49. 49. “IS THE EVM EFFICIENT ENOUGH?”
  50. 50. BLOCKCHAIN PLATFORMS AS A COMPUTER Ethereum Virtual Machine (EVM) Poor development environment support Confusing language syntax Inefficient Gas system (The Extensibility Issue) Language compatibility Resource-intensive; not suitable for embedded systems
  51. 51. GEMER: AN OPEN SOURCE, DISTRIBUTED RUNTIME
  52. 52. THANK YOU Further inquiries: me@unifiedh.com Personal GitHub: github.com/unifiedh

×