MIPI DevCon 2016: Image Sensor and Display Connectivity Disruption
•
1 like•1,553 views
The ability to leverage mobile technologies into new consumer, medical, industrial, and automotive markets creates challenges in image sensor and display interfacing. When interface types or number of the interfaces do not match between image sensors, displays and processors, a bridge is required to enable such connectivity. In this presentation, Grant Jennings of Lattice Semiconductor describes connectivity through programmable interface bridges to aid in the development of these systems that were unforeseen or previously could not be rationalized.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to MIPI DevCon 2016: Image Sensor and Display Connectivity Disruption
Similar to MIPI DevCon 2016: Image Sensor and Display Connectivity Disruption (20)
More from MIPI Alliance
More from MIPI Alliance (20)
Recently uploaded
Recently uploaded (7)
MIPI DevCon 2016: Image Sensor and Display Connectivity Disruption
- 1. Image Sensor and Display Connectivity Disruption Use Case Trends from a Continually Evolving Mobile Ecosystem Grant Jennings Lattice Semiconductor
- 2. Agenda • Image Sensor and Display Evolution • Influence of Mobile on Horizontal Markets • Permutations of Interface Standards • Enabling Innovation Through Bridging • Image Sensor and Display Use Case Challenges • Bridging Use Case Examples • Enabling Innovation Through Programmability • Future Considerations 2
- 3. 3 2-5 Cameras 1-2 Displays Cost/Power/Size Critical 1-4 Display Interfaces 0-16 Camera Interfaces Varying Requirements Difficult to find the right logic devices to accommodate all applications and ideas Image Sensor and Display Evolution Influence of Mobile on Horizontal Markets
- 4. 4 SmartphonePre-Smartphone Source Synchronous CMOS CSI-2 MIPI DSI D-PHY MIPI DPI RGB PROPRIETARY MIPI DBI GPIF SubLVDS HiSPi SLVS PROPRIETARY Larger Systems Embedded Clock SLVS-EC FPD-LINK III eDP PROPRIETARY SDI ETHERNET M-PHY Difficult to find build IP/interface knowledge base Image Sensor and Display Evolution Permutations of Interface Standards C-PHY LVDS
- 5. Agenda • Image Sensor and Display Evolution • Influence of Mobile on Horizontal Markets • Permutations of Interface Standards • Enabling Innovation Through Bridging • Image Sensor and Display Use Case Challenges • Bridging Use Case Examples • Enabling Innovation Through Programmability • Future Considerations 5
- 6. 6 Resolving unforeseen issues Enabling backward compatibility Solving new problems Bridge IC DSLR PROCESSOR Sub-LVDSIMAGE SENSOR D-PHY Incompatible inputs MCU SPI D-PHY DISPLAY Incompatible outputs Bridge IC APPLICATION PROCESSOR D-PHY DUAL D-PHY DISPLAY Insufficient outputs D-PHY D-PHY MOBILE PROCESSOR D-PHY Insufficient inputs RightEyeLeftEye Bridge IC Bridge IC Enabling Innovation Through Bridging
- 7. 7 Camera #2 Camera #N . . . Processor Reference Clock MIPI CSI-2 MIPI CSI-2 MIPI CSI-2 MIPI CSI-2 I2C I2C I2C I2C Sync Sync Sync APPLICATIONS Enabling Innovation Through Bridging Multiple Cameras Bandwidth Aggrega.on, Frame Synchroniza.on, Virtual Channels, I2C Addressing Bridge Camera #1
- 8. Enabling Innovation Through Bridging 8 Bridge Processor Camera Proprietary Interface MIPI CSI-2 I2C SPI Readout Control APPLICATIONS Proprietary/Custom Bridging Frame Readout Control, I2C to SPI, Bandwidth Aggrega.on, Pixel vs. Byte Serializa.on
- 9. 9 DSI to SPI/CMOS SPI to DSI CSI-2 to SPI/CMOS Sensor Bridge uP Object Detection SPICSI-2 Display Bridge Frame Data Display Controller DSISPI uP Specialty Display Bridge CMOSDSI AP/uP APPLICATIONS Enabling Innovation Through Bridging Low Speed Peripheral Interfacing Bandwidth Reduc.on Through Co-Processing, Master/Slave, Customized Interfacing
- 10. 10 AP DSI DSI DSI Display Display X Gbps X/2 Gbps X/2 Gbps Specialty Image Processor DSI DSI CSI-2 Display AP, Bridge, etc. X Gbps X Gbps X Gbps Display DSI CSI-2 AP CSI-2 Camera APPLICATIONS Bridge Bridge Bridge Enabling Innovation Through Bridging Splitters or Bandwidth Reducers SpliIng, Configurable Interface IP, Cropping/Scaling, Clock Domain Crossing
- 11. Enabling Innovation Through Bridging 11 Applica:on Processor Bridge DCS Commands for New Display DSI DSI Bridge RAW to YUVCSI-2 CSI-2 Cropping Crop/Scale Packet Repair Continuous Clock APPLICATIONS Display Applica:on Processor Camera Configura.on/Control, Data Type Conversion, Short Packet Repair, Enabling Termina.on Repeaters
- 12. Agenda • Image Sensor and Display Evolution • Influence of Mobile on Horizontal Markets • Permutations of Interface Standards • Enabling Innovation Through Bridging • Image Sensor and Display Use Case Challenges • Bridging Use Case Examples • Enabling Innovation Through Programmability • Future Considerations 12
- 13. COST POWER PERFORMANCE SIZE 13 CSI-2 DSI RAW YUV YCbCr I2C SPI I3C D-PHY C-PHY SLVS HiSPi 8bpp 10bpp 12bpp 14bpp 18bpp 24bpp HS signaling DPCM Virtual Channels Embedded Data Encoding 1 lane 3 lane 2 PHYs RAW 1 Logic Device Clocking scheme Developing a bridge IC for a singular use case often not cost effective Hard to find silicon for non-conforming cases Hard to develop IP and architectural knowledge 1000s of permutations
- 14. IMAGE SENSOR CAMERA INTERFACE BRIDGE DISPLAY INTERFACE BRIDGE IMAGE SENSOR BRIDGING EXAMPLES DISPLAY SENSOR BRIDGING EXAMPLES APPLICATION PROCESSOR 1:2 Display Splitter Bridge APPLICATION PROCESSOR 1:2 High-Resolution Display BridgeHigh-Resolution Camera Bridge APPLICATION PROCESSOR Programmable Bridge Programmable Bridge Programmable Bridge Programmable Bridge Programmable Bridge Programmable Bridge APPLICATION PROCESSOR APPLICATION PROCESSOR Display Display Display Display Single Programmable Video Bridge Handles Many Use Cases With Single Device 14
- 15. FPGAs ASIC / ASSPs Time-to-Market Cost Flexibility Performance Power/Size The pASSP Continuum The Best of Both Worlds 15
- 16. 16 Programmable IO RX: D-PHY / SubLVDS / LVDS / SLVS200 / CMOS TX: LVDS / CMOS 1.2 Gbps/Lane 14 IO / 7 Pairs MIPI D-PHY 6 Gbps RX & TX 4 data lanes 1 clock lane MIPI D-PHY 6 Gbps RX & TX 4 data lanes 1 clock lane GPIOs I2C/SPI Programmable FPGA Fabric 5,936 LUTs 180 kbits block RAM 47 kbits distributed RAM Enough FPGA resources to handle video: Muxing Merging Demuxing Arbitration Splitting Data Conversion Custom Protocol Design Programmable IO RX: D-PHY / SubLVDS / LVDS / SLVS200 / CMOS TX: LVDS / CMOS 1.2 Gbps/Lane 16 IO / 8 Pairs Power Management Unit CrossLink Must have right amount and types of interfaces Must have right amount of programmability Example – Lattice CrossLink™ Block Diagram Problem 1 – Having the right device
- 17. 17 Display IPs Camera IPs DSI to DSI Dual CSI-2 to CSI-2 DSI to Dual DSI CSI-2 to Parallel CMOS Dual DSI to Dual DSI Parallel CMOS to CSI-2 DSI to LVDS SubLVDS to CSI-2 DSI to Dual LVDS Dual DSI to Dual LVDS DSI to DPI Single LVDS to DSI Dual LVDS to DSI Dual LVDS to Dual DSI DPI to DSI Must have end-to-end solutions for common use cases IPs and Starting Points Problem 2 – Having the Right IP Configurable Data Types (RAW, YUV, RGB, YCbCr, Custom, Bits per Pixel) Configurable Number of Data Lanes and PHYs Adjustable Input & Output Frequencies DCS Command ROM IP Customization
- 18. 18 36 WLCSP 64 ucfBGA 81 csfBGA 80 ctfBGA 0.4 mm pitch 0.4 mm pitch 0.5 mm pitch 0.65 mm pitch 2.46 x 2.46 mm 3.5 x 3.5 mm 4.5 x 4.5 mm 6.5 x 6.5 mm 36 WLCSP 80 ctfBGA Can’t pay a penalty for using programmable logic Small Form Factor Problem 3 – Power, Cost, Size
- 19. Today’s Bridging Devices Need to be Better than Stand-Alone ASSPs 1X Quicker Time to Market Custom solutions developed in real time Same Low Power Optimized use cases Same Behavior Firmware retention in the device Smaller Form Factor WLCS and flip chip packages Same Low Cost Pad limited not gate limited ASSP RAW, YUV 8, 10, 12 bpp 1-lane, 2-lane, 4-lane Continuous, Non- Continuous pASSP Firmware #1 YUV, 8 bpp, 1-lane Firmware #2 RAW, 16 bpp, 4-lane Firmware #2 JPG, 8 bpp, 2-lane CFG vs. 19 Example of ASSP Versus pASSP Paradigms:
- 20. Agenda • Image Sensor and Display Evolution • Influence of Mobile on Horizontal Markets • Permutations of Interface Standards • Enabling Innovation Through Bridging • Image Sensor and Display Use Case Challenges • Bridging Use Case Examples • Enabling Innovation Through Programmability • Future Considerations 20
- 21. Future Considerations 21 • Ecosystem standards enable innovation • Cost, power, size • Brings commonality to invoke new uses • FPGA programmability enables standards adoption • Reduces risks with backward compatibility • FPGA programmability enables innovation • New and unforeseen use cases made possible • FPGA programmability is inexpensive • With the right architecture, same as ASSP • Help FPGA programmability help you • Working together though MIPI to enable the next wave of use cases
- 22. Thank You Connecting Everything to Anything…. 22
- 23. 23