MIPI DevCon 2016: Mobile User Interface Aggregation for Heterogeneous Compute Intensive Solutions
•
0 likes•213 views
A few years ago, buttons were the only user interface available to control a mobile device. Now you can use touch screens in addition to, or even in replacement of, buttons. Voice commands are also used and we’re seeing more development on gesture technologies and recognition of facial expressions. With immediate feedback expectations, the burden on analyzing and routing sensor data quickly becomes more cumbersome. In this presentation, Lattice Semiconductor's Abdullah Raouf discusses solutions supporting these MIPI interfaces, as well as solutions required for compute-intensive applications. He also describes the value of FPGAs on repetitive computational needs, and how FPGAs will improve system power efficiency.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to MIPI DevCon 2016: Mobile User Interface Aggregation for Heterogeneous Compute Intensive Solutions
Similar to MIPI DevCon 2016: Mobile User Interface Aggregation for Heterogeneous Compute Intensive Solutions (20)
More from MIPI Alliance
More from MIPI Alliance (20)
MIPI DevCon 2016: Mobile User Interface Aggregation for Heterogeneous Compute Intensive Solutions
- 1. Mobile User Interface Aggregation for Heterogeneous Compute Intensive Solutions Abdullah Raouf Lattice Semiconductor
- 2. Agenda • Data Capture • Sensors available • Use case mapping • Data Transfer • MIPI interface options • Value of using FPGA to transfer data via MIPI interfaces • Data Analysis • Analytics trends • MHC architectures • MHC solutions used every day • FPGA Architecture & Solutions 2
- 3. Always On, Always Aware 3
- 4. Capturing Data Audio Sensors: Voice recognition, phrase detection, location sensing 4 Image Sensors: Gesture, eye tracking, proximity, depth, movement Power efficient heterogeneous compu2ng needed for sensor s2tching and real-2me feedback Health Sensors: EKG, EEG, EMG, temp Other Sensors: Gyro, compass, accelerometer, magnetometer, light Re al time sensor feedba ck Al w ays aware and worki ng Iris ▪ Fingerprint ▪ M otion▪Image▪EKG▪Gyro ▪Audio▪Accelerometer▪E EG ▪Magnetometer▪Gestur e ▪ EMG ▪
- 5. A Variety of Interface Needs Exist 5 analog DSI I2S YUV SLVS I2C SPI I3C D-PHY C-PHY SLVS-EC HiSPi CSI-2 PCM sLVDS PDM SLIMbus Virtual Channels RFFE UART RAW proprietary analog PDM SoundWire SLIMbus I2S D-PHY YUV RGB SPI C-PHY DSI CSI-2 sLVDS SLVS I3C PCM
- 6. Bridge/Aggregate Easily with FPGAs 6 SOC / AP SoundWire iCE Family PDM I2S UART I2C/SPI I3C CSI-DSI SLVS LVDS 121 balls 36 balls 16 Balls 80 balls 1.4mm x 1.4mm 6mm x 6mm
- 7. MIPI Transfer Optimization 7 … Microphones Speakers Displays Cameras SOC / APSensors Use Case MIPI Standard Display MIPI DSI Image sensor MIPI CSI-2 Generic sensor aggrega2on SPI, I2C, UART, GPIO ! MIPI RIO, MIPI I3C Audio MIPI SoundWire, MIPI SLIMbus RF MIPI RFFE BaWery MIPI BIF
- 10. TradiHonal Algorithms vs Neural Networks TradiHonal Algorithms • Based on pre-conceived events • Used to find exact or approximate sol’n • Domain exper2se required • Architecture & data flow explicitly defined • Func2onally comprehensible • Implemented in SW Neural Network • Based on experience probabili2es • Used to find paWerns in data • Structure and data flow evolves within the network based on training • Can be implemented in SW (GPUs) or HW (DSPs and FPGAs)
- 11. Basic Neural Network Each “neuron” is multiply-accumulate Mul2ply Add/Accumulate
- 12. Basic Neural Network GPUs use serial approach Mul2ply Add/Accumulate GPU Weight Memory
- 13. Basic Neural Network FPGAs are parallel using local memory FPGA LUTS DSP EBR Mul2ply Add/Accumulate
- 14. Enabled by: " Moore’s Law " Voltage scaling Constrained by: × Power × Complexity Enabled by: " Moore’s Law " Voltage scaling Constrained by: × Power × Parallel SW × Scalability Enabled by: " Abundant data parallelism " Power efficiency " Repe22ve computa2on Constrained by: × Programming tools/models × High cost of entry Programmability Repe22ve Computa2onal Quickness Enabled by: " Time to market " Flexible I/Os " Accelerators " Parallel processing " Power efficiency " Repe22ve computa2on Constrained by: × Programming tools Mobile Heterogeneous CompuHng Architectures Micro-Processor Advancement Single-Core Processor Mul2-Core Processor Heterogeneous Processor HETEROGENEOUS COMPUTING FPGA MHC HOMOGENEOUS COMPUTING
- 15. CNN Hardware Energy Efficiency FPGAs win in published results
- 16. Complex Functions Always-on sensor data Acous2c beam forming Feature Rich >5K LUTs >6x DSPs >1Mb RAM FPGA Range of Mobile SoluHons Simple Functions MIPI RFFE antenna tuning, RGB LED, Level shieing, etc. Mid Range Functions Pedometer, FFTs, co-processing, etc. Mid Range Feature Set >3K LUTs >2x DSPs >100 Kb SRAM Simple Feature Set Sub 2K LUTs Sub 80Kb RAM 1.4 x 1.4mm 2.08 x 2.08mm >21 IOs 10 IOs 2.15 x 2.55mm >21 IOs 2.5 x 2.5mm >21 IOs Machine Vision 4K video transfer with MIPI D-PHY High Speed I/O >5K LUTs >6x DSPs Hard D-PHY
- 17. Summary FPGAs are changing the way you interact with your devices • Energy-efficient parallel processing for repe22ve number crunching needed for MHC • Embedded memory enables implementa2on of always-on sensor data buffers • Supports a variety of bridging, buffering and display applica2ons (Flexible I/Os) • Accelerate key innovaHons in next-genera2on mobile and industrial products • Carpe Datum aka seize the data • Data has no value if it is not captured, transferred and analyzed correctly • Use MIPI I/O and Lajce pre-processing solu2ons to truly seize value out of data!