Tom Watzka and Satwant Singh of Lattice Semiconductor highlight the evolution of innovative applications in mobile-influenced markets, including the unique issues faced by system and software developers. Examples of end applications and extrapolated architectural trends for several use cases will also be explored.

1. Tom Watzka Satwant Singh
Lattice Semiconductor Lattice Semiconductor
Mobile Influenced Markets –
Evolution of Camera and
Display Uses

2. © 2017 MIPI Alliance, Inc.
• The PC industry created demand in
adjacent markets such as test
equipment, POS terminals, and
industrial controls
• Designers leveraged their access to
standardized APIs and hardware
• In a similar way, the mobile industry is
experiencing demand for high quality,
low cost components in mobile
influenced markets
2
Lattice Semiconductor
Lattice MIPI D-PHY℠ Applications

3. © 2017 MIPI Alliance, Inc.
Traditional Display Applications
3
Lattice Semiconductor
Rear
Display
Front
Display
SplitAP
AP
• Traditional display applications
will include one (maybe two)
static displays

4. © 2017 MIPI Alliance, Inc.
Industrial
4
Lattice Semiconductor
Appliances
Automotive
SOC Bridge
AP Bridge LVDS
DSI
LVDS
BRIDGING
Many consumer, automotive, and industrial
applications use older LVDS interfaces

5. © 2017 MIPI Alliance, Inc.
AR/VR (Displays)
5
Lattice Semiconductor
SplitAP
Left Eye Right Eye
HDMI Bridge
Input
Monitor
Minimum frame
rate of 75 Hz is
needed to avoid
motion sickness.
Merging is the
favored method
to minimize eye-
to-eye latency.

6. © 2017 MIPI Alliance, Inc.
Mux vs Merge
6
Lattice Semiconductor
MergeLine 1 Line 2Camera 1
Output
Line 1
Line 1 Line 2Camera 1
Line 2
Input Output = 2X Input Bandwidth, 0 Latency
MuxFrame 1Camera 1
Output Frame 1
Frame 1Camera 2
Frame 2
Frame 2
Frame 3
Frame 3
Frame 4
Frame 4
Frame 2 Frame 3 Frame 4
Input Output = 1X Input Bandwidth, 1 Frame Latency
Frame 1 Frame 2 Frame 3
…
• Video data can be “multiplexed” through a single MIPI D-PHY
port by mux-ing frame by frame, or merging to super frames.

7. © 2017 MIPI Alliance, Inc.
Traditional Camera Applications
7
Lattice Semiconductor
Rear
Camera
Front
Camera
Mux AP
• Traditional camera applications
statically switch from one camera
to the other

8. © 2017 MIPI Alliance, Inc. 8
Lattice Semiconductor
8
Requires Environment Awareness
• IMU – Inertial Measurement Unit Performing Sensor Fusion for Accelerometer,
Gyrometer & Magnetometer
• Cameras
– Distance Measurement
– Environmental Understanding
AR/VR (Cameras)

9. © 2017 MIPI Alliance, Inc.
AR/VR Tracking
9
Lattice Semiconductor
Outside-In System
• Two approaches to positional tracking:
– Outside-In requires external hardware
– Inside-out is self contained
Inside-Out System
(such as Microsoft Hololens)

10. © 2017 MIPI Alliance, Inc.
AR/VR Permutations for Inside-Out
10
Lattice Semiconductor
2X Bandwidth, 0 Latency
~3X Bandwidth, 0 Latency
:
Input
~5X Bandwidth, 0 Latency
TOF TOF
TOFTOF
Output
• Inside-out
implementations are
growing in the number
of cameras and sensors

11. © 2017 MIPI Alliance, Inc.
Drones
11
Lattice Semiconductor
• Require same things as AR/VR
• Plus more cameras
360 Cameras
Automotive
• Just more cameras
Merge Transport GPU
• Require same things as AR/VR
• Plus transport
Merge AP
Merge AP

12. © 2017 MIPI Alliance, Inc. 12
Lattice Semiconductor
Everything Else
SOC Bridge
AP Bridge LVDS
DSI
LVDS
LVDS
DSI
AP Bridge
Dual
LVDS
DSI
LVDS
LVDS
SOC Bridge
CMOS
RGB
/
24
DSI
APBridge
CSI-2
CMOS
RGB
/
24
SOCBridge
CSI-2
CMOS
RGB
/
24
APRe-Time
CSI-2CSI-2

13. © 2017 MIPI Alliance, Inc.
• FPGAs have general purpose IOs in
addition to specialized IOs such as
MIPI D-PHY
• FPGAs provide an array of flexible
programmable logic cells that can
support high speed data flow
including splitting and merging
functions
13
Lattice Semiconductor
CrossLink
Programmable I/O
MIPI D-PHY
Programmable FPGA Fabric
Why FPGAs? Flexibility

14. © 2017 MIPI Alliance, Inc.
Why FPGAs? Time To Market
14
Lattice Semiconductor
ASICs = 18 month
dev cycle
Consumer =
6 – 12
month dev
cycle
Design RTL
Simulate
Pass
Synthesize
Debug
Pass
Features
Time
Days to
Weeks
Dev Cycle
• Product development cycle is much faster than ASIC cycles
• FPGAs enable innovation

15. © 2017 MIPI Alliance, Inc.
Summary
• MIPI components have become ubiquitous
– Cameras, Displays, APs, Accelerometers, Gyrometers, Magnetometers …
• Mobile influenced markets are leveraging these components
in all sorts of ways and combinations
• FPGAs have been instrumental in enabling these new (and
unforeseen) markets in ways that simple bridges cannot, in
terms of time and functionality
15
Lattice Semiconductor